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Chapter  I 
INTRODUCTION 

Education  is  exclusively  concerned  with  behavior 
change  for  the  purpose  of  developing  and  maintaining  complex 
repertoires  of  culturally-valued  human  behavior.   In  order 
to  evaluate  the  effectiveness  of  its  procedures,  education 
must  have  a  reliable  and  sensitive  method  for  describing 
and  measuring  the  behavior  changes  it  produces.   Currently, 
such  evaluation  is  performed  almost  totally  by  psychometry  — 
psycho-educational  tests  and  rating  scales.   Despite  its 
undoubted  importance  historically,  psychometry  has  new  been 
shown  to  have  serious  shortcomings  of  both  an  empirical 
and  ethical  nature.   These  deficits  have  been  examined  in 
great  detail  by  Hoffman  (1362,  1565,  1967).   Hoffman's 
documented  discussions  include  statistical  misuse  in  the 
development  of  tests,  misuse  of  test  results,  fallacies 
in  the  use  of  pre-tssts,  as  well  as  the  corrupting  effects 
tests  have  on  educational  practice. 

Despite  these  deficits,  educators  continue  to 
use  psycho-educational  tests  and  rating  scales  in  most 
evaluation  research,  apparently  because  they  are  unaware  of 
more  acceptable  alternatives.   Tests  and  rating  scales 
are  used  to  evaluate  beneivior  changes  in  children,  teachers, 
and  school  administrators,  as  well  as  to  assess  the  effects 
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of  curricula  and  teaching  methods  on  behavior.  With  the 
partial  exception  of  achievement  tests,  psychometric 
procedures  never  directly  measure  the  behavior  in  question. 
Unfortunately,  the  trend  in  recent  years  would  seem  to  be 
away  from  more  simple  and  direct  measures  of  behavior  and 
toward  procedures  which  rely  highly  on  indirect  measurement 
and  a  complex  chain  of  inferential  statements  mediated  by 
hypothetical  constructs.   The  Illinois  Test  of  Psycholinguistic 
Abilities  is  an  example  of  this  trend. 

The  present  dissertation  seeks  to  utilize  free 
operant  technology,  in  particular  the  direct  and  continuous 
recording  of  pupil  performance,  to  evaluate  and  analyze  a 
programed  instruction  curriculum  material.   At  the  same  time, 
the  records  gathered  can  be  used  to  further  understand  free 
operant  technology  itself.   Therefore,  the  dissertation  will 
describe  not  only  human  behavior  change  as  a  function  of 
the  programed  material,  but  it  will  describe  the  effects 
of  the  technology  on  pupil  performance  rates.   The  findings 
of  this  t"pe  of  research  are  directly  applicable  to  children 
and  may  be  directly  related  to  further  applications  in 
which  the  evaluation  of  educational  procedures  is  of  concern. 

Related  Research 
The  Evaluation  oj!  Programed  Instruction 

A  review  of  the  literature  relating  to  the  evaluation 
of  programed  instruction  reveals  that  there  is  no  generally 
accepted  method  available.   There  are,  however,  several 
methods  which  are  used  with  some  frequency. 


The  first  of  these  is  the  checklist.   Although 
Newman  (1965)  has  recommended  against  the  development  and 
use  of  checklists  without  first  having  conducted  exhaustive 
research  on  their  reliability  and  validity,  the  checklist  has 
nevertheless  become  the  most  popular  and  convenient  method 
of  evaluating  programed  instruction. 

Before  1963,  checklists  were  commonly  created  by 
individual  researchers  for  their  own  purposes  (Fry,  1963; 
Hughes,  1962).   Most  of  these  checklists  were  quite 
heterogeneous,  and  rested  more  on  considered  opinion  and 
theoretical  orientation  than  on  empirical  research. 

In  1962,  The  Joint  Committee  for  Criteria  for 
Assessing  Instructional  Programs  began  publishing  checklists 
for  use  in  evaluating  programed  instruction.   These  lists 
were  revised  yearly  by  the  committee,  on  the  basis  of 
their  demonstrated  utility  in  the  applied  situation.   The 
absence  of  systematic  research  in  evaluating  checklists 
produced  an  instrument  that  was,  at  best,  crude. 

Other  checklists  were  later  developed  (Jacobs,  Maier, 
and  Stolurow,  1966),  but  in  the  final  analysis,  the  Joint 
Committee's  checklist  has  become  the  standard  in  the  field.   It 
has  been  endorsed  by  both  the  national  Society  for  the  Study  of 
Education  (NSSE)  and  the  Division  of  Audio-visual  Instruction  (DAVI) 
of  the  National  Educational  Association,  the  two  organizations 
most  concerned  with  quality  standards  in  education. 

Ironically  enough,  the  chief  rival  of  the  Joint 
O  mmittee's  checklist  is  another  well-respected  educational 
standard  that  was  not  developed  specifically  for  evaluating 


instruction.   Bloom's  Taxoncmy  of  Educational  Objectives 
(1956)  has  been  repeatedly  invoked  as  the  evaluative 
standard  form  a  "cognitive"  point  of  view.   The  chief 
spokesman  for  this  type  of  evaluation  is  Louise  Tyler 
(1966) .   Not  satisfied  with  theory  only,  she  has  reported 
some  data  on  its  use  by  teachers  already  familiar  with  the 
taxonomy.   Newman  (1965)  has  used  it  to  evaluate  programed 
instruction  in  the  Social  Studies. 

The  second  method  of  evaluation  that  can  be 
distinguished  might  more  properly  be  called  the  comparison 
procedure.   It  generally  takes  the  form  of  comparing 
programed  instruction  with  traditional  teaching  methods, 
or  programed  texts  with  standard  texts. 

Schramm  (1964)  indicates  that  much  cf  the 
evaluative  research  done  in  programed  instruction  is  of 
this  nature.   An  examination  of  Educational  Abstracts 
confirms  this  statement  for  the  subsequent  years.   The 
difficulty,  however,  is  that  comparison,  as  a  research 
method,  reveals  little  or  nothing  about  the  programed 
instructional  material  per  se.   Nevertheless,  the  Joint 
Committee  recommends  comparison  as  a  method  for  the  external 
validation  of  programed  material,  and  it  may  be  an  acceptable 
procedure  for  this  task. 

Another  difficulty  with  comparison  studies  lies 
in  the  nature  of  the  research  design  employed.   It  is  almost 
impossible,  in  this  type  of  research,  to  control  for 
individual  differences  in  children  and  in  teachers,  and 
its  even  more  difficult  to  control  for  differences  in  content. 
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The  majority  of  research  of  the  comparative  type,  as  Schramm 
notes,  is  so  poorly  done  that  little  faith  can  be  placed 
in  the  validity  of  the  results.   There  are  fortunate  exceptions, 
particularly  with  regard  to  exceptional  child  populations. 
Blackraan  and  Copobianco  (1965) ,  for  example,  report  on 
the  use  of  a  specific  programed  material  with  retarded 
children.   Likewise,  Rainey  and  Kelly  (1967)  report, 
the  use  of  a  time-telling  program  with  educable  retardates, 
and  Strong  (1964)  reports  evaluating  a  program  with  deaf 
opulations.   This  research  is  child-oriented  and  involves 
the  determination  of  the  utility  of  a  specific  program  for 
developing  a  defined  behavior  in  a  given  exceptional  child 
population.   The  utility  of  this  type  of  research  for  the 
practicing  classroom  teacher  should  not  be  under-estimated. 
Closely  related  to  the  programed  instruction  with 
"other"  comparison,  is  the  programed  instruction  with 
achievement  test  comparison.   Normally,  the  test  used  is 
one  of  the  standard  achievement  tests  such  as  the  Wide  Range 
Achievement  Test  or  the  Metropolitan  Achievement  Test.   In 
this  case,  the  research  questions  are  directed  to  difference 
scores  on  the  specific  test  before  and  after  the  administration 
of  the  programed  instruction  material.   The  discontinuous 
(before  and  after)  nature  of  this  procedure  is  a  serious 
shortcoming  because  it  does  not  allow  a  point  to  point 
analysis  of  the  relationship  between  the  program  and  the 
child's  behavior.   This  method,  like  the  programed  instruction 
with  "other"  comparison,  is  recommended  by  the  Joint 
Committee:  for  the  external  validation  of  the  material. 
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The  careful  reader  of  research  is  quick  to  note 
that  the  dependent  variable  in  these  studies  is  not  programed 
instruction  performance,  but  achievement  test  performance. 
Programed  instruction  performance  is  then  inferred  from 
the  test  performance. 

In  his  text  on  evaluating  programed  instruction 
Jacobs,  ot  al . ,  (1966),  mentions  the  Denver  Study  as  the  classic 
model  for  evaluation.   This  study,  reported  iri  toto_  by  Jacobs, 
utilized  both  the  programed  instruction  with  "other"  comparison 
and  the  programed  instruction  with  achievement  test  types 
of  evaluation.   The  questions  asked  in  this  study  were: 

1)  Do  classes  taught  by  the  program  only, 
by  a  teacher  only,  and  by  a  combination 
of  a  teacher  plus  the  program  differ  in 
the  outcomes  of  levels  of  achievement, 
attitudes  toward  programed  instruction 
and  interest  in  Spanish? 

2)  How  are  the  input  characteristics  of  initial 
achievement,  academic  aptitude,  and  attitude 
towards  Spanish  related  to  the  outcomes 

of  achievement  in  interest  in  Spanish  in 
each  instructional  group?   For  example,  do 
the  brighter  classes  learn  more  than  the 
slower  ones  in  each  group? 

3)  Are  teacher's  attitudes  towards  Spanish 
and  various  teaching  methods  related  to 

the  instructional  methods  used  in  the  study? 

A  careful  examination  of  these  questions  reveals 
that  in  no  case  is  programed  instruction  performance  the 
dependent  variable.   This  study,  the  apparent  classic  in 
the  evaluation  of  programed  instruction,  is  in  fact  not 
an  evaluation  of  the  internal  aspects  of  programs.   Because 
it  is  an  evaluation  about  programed  instruction  as  it 
relates  tovarious  dependent  variables  such  as  achievement 
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test  and  attitude  scale  scores,  no  direct  statement  may 

be  made  regarding  the  behavior  change  on  the  program  itself. 

Also  mentioned  in  the  evaluation  literature, 
but  clearly  not  research,  is  a  caution  to  the  program 
user  to  check  the  credentials  of  the  program  author  and 
the  publisher.   The  Joint  Committee  advises  all  publishers 
to  include  with  each  program  sold,  complete  developmental 
and  utilization  testing  data.   The  advisement,  in  practice, 
is  little  heeded. 

In  summary,  there  are  four  main  methods  of  evaluating 
programed  instruction  material.   These  are: 

1)  the  checklist, 

2)  Bloom's  Taxonomy  of  Educational  Objectives, 

3)  programed  instruction  with  "other"  comparison, 

4)  programed  instruction  with  achievement  test 
evaluation. 

It  is  clear  from  this  review  that  programed 

instruction  performance  is  typically  not  the  dependent 

variable  in  the  evaluation  of  programed  instruction 

materials. 

Free  Operant  Research  in  Education 

In  the  history  of  phycho.logy,  the  use  of  free 
operant  techniques  in  the  analysis  of  human  behavior  is 
relatively  recent.   It  was  only  in  19-19  that  Fuller  demonstrated 
experimental  control  over  a  vegetative  mental  retardate. 
Skinner's  text,  using  knowledge  acquired  in  the  study  of 
free  operant  behavior  of  animals  to  describe  human  behavior, 
appeared  in  1953.   In  it,  an  empirical  framework  for  the 


3 
experimental  analysis  of  human  behavior  was  presented,   Sound 
experimental  data  were  not  forthcoming  until  Lindsley's  classic 
study  with  chronic  psychotics  (1956) .   Skinner  (1958) ,  reporting 
on  his  research  with  programed  instruction  technology,  an 
extension  of  free  operant  techniques  with  animals,  excited 
much  interest  in  the  educational  community.   Bijou  (1957,  1958) 
developed  observation  techniques  for  young  children  patterned 
closely  after  those  used  by  Lindsley. 

The  marriaga  between  the  educator  and  free  operant  methods 
was  not  long  in  coming.   Birnbrauer,  Bijou,  Wolf,  and  Kidder 
(19S5)  demonstrated  the  application  of  free  operant  techniques 
in  a  classroom  situation  using  programed  instruction  as  a 
curricular  core.   Zimmerman  and  Zimmerman  (1962)  also  applied 
free  operant  techniques  in  a  classroom  with  much  less  structure 
than  Birnbrauer' s  classroom. 

Ayllon  and  Azrin  (1964)  demonstrated  the  functional 
utility  of  token  economies  in  shaping  the  behavior  of  patients  in 
a  mental  hospital.   Girardeau  and  Spradlin  (1964)  used  the 
same  type  of  token  control  with  retardates  at  the  Parsons 
State  Home  and  Training  School. 

At  that  time,  however,  there  was  no  systematic  or 
standard  method  in  human  free  operant  research.   Although  a 
precise  language  was  available  for  the  description  of  animal 
behavior  (Ferster  and  Skinner,  1957),  its  application  to  the 
human  situation  was  confusing  and  left  much  to  be  desired. 
Research  reports  were  presented  in  the  literature  with  method 
and  discussion  sections  so  radically  non-standard  that  scientific 
replication  (Sidman,  1964)  was  virtually  impossible. 
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0.  R.  Lindsley  (1964)  put  forth  a  numerical- 
temporal  descriptive  language.   Its  purpose  was  to  precisely 
describe  behavior  and  those  events  related  to  behavior, 
either  in  number  or  in  tine.   This  was  followed  by  a 
major  strategy  statement  indicating  that  behavior  change 
must  be  produced  by  teachers  and  parents  in  order  to  meet 
the  existent  need  (Lindsley,  1968).   Lindsley  (1966) 
has  coined  the  term  precision  teaching  to  describe  the  use 
of  free  operant  methods  by  teachers.   The  details  of 
precision  teaching  and  the  descriptive  language  are  available 
elsewhere  (Koenig,  1967;  Caldwell,  1967;  Haughton,  1967) 
and  will  not  be  treated  at  length  here. 

Although  rate  was  the  accepted  datum  unit  for 
animal  free  operant  research,  researchers  using  human 
subjects  felt  little  obligation  to  follow  suit.   Instead, 
many  relied  on  the  more  standard  educational  datum  units 
of  absolute  number  and  percent.   In  a  study  of  the 
sensitivity  of  the  various  datum  units  to  behavior  change, 
Kolzschub  and  Dobbs  (1966)  demonstrated  that  rate  was 
consistently  more  sensitive  than  other  units  examined. 
Subsequent  research  (Caldwell,  1966;  Johnson,  1967b)  has 
supported  this  finding. 

Free  operant  techniques  in  general,  and  precision 
teaching  in  particular,  have  found  wide  acceptance  in  special 
education  (Haring  and  Schiefelbusch ,  1967).   Operant  techniques 
have  been  used  almost  exclusively  with  single  organisms. 
Because  special  educators  have  long  been  tailoring  curricular 
programs  for  individual  children,  the  techniques  seemed 
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ideally  suited  for  that  area  of  education  concerned  primarily 
with  individual  differences. 

Koenig  (1967)  used  precision  teaching  in  a 
classroom  setting  with  emotionally  disturbed  children  to 
examine  a  wide  range  of  academic  and  disruptive  behaviors. 
Haughton  (1966)  demonstrated  the  functional  utility  of 
direct  and  continuous  recording  of  behavior  compared  with 
an  examination  of  achievement  tests  in  predicting  pupil 
performance.   Johnson  (1967c)  demonstrated  dramatically 
that  achievement  tests  and  pupil  performance  on  similar 
material  generated  different  performance  rates  when  both 
pupil  performance  and  achievement  test  performance  were 
directly  recorded.   Johnson  (1967a)  found  that  teacher- 
planned  rates  (number  of  problems  assigned  by  the  teacher 
divided  by  the  number  of  minutes  allotted  by  the  teacher 
for  their  solution)  were,  in  part,  determiners  of  subsequent 
pupil  performance  rate. 

Johnson  (1969)  is  currently  engaged  in  the  use 
of  precision  teaching  to  evaluate  the  Science  Research 
Associates  arithmetic  series.   This  study,  as  yet  unpublished, 
is  the  only  reference  known  to  this  writer  on  the  evaluation 
of  any  curriculum  through  the  use  of  free  operant  techniques. 

In  summary,  free  operant  techniques  originally 
used  exclusively  in  the  study  of  animal  behavior,  have,  in 
recent  years,  seen  wide  application  to  human  performance. 
When  used  in  classroom  settings,  the  techniques  may  be 
collectively  referred  to  as  precision  teaching  (Lindsley, 
1966).   Although  precision  teaching  has  been  applied  to 
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a  broad  range  of  educational  problems,  only  in  one  case 
has  it  been  used  in  the  analysis  of  curricular  material. 

Statement  of  Purpose 
The  purpose  of  this  dissertation  is  to  assess 
the  usefulness  of  free  operant  technology  (precision 
teaching)  in  the  analysis  of  programed  curricular  materials. 

Method 
Subjects 

Fifty-nine  children,  thirty-seven  boys  and 
twenty-two  girls  from  two  Alachua  County,  Florida,  elementary 
schools,  Stephen  Foster  Elementary  and  Duval  Elementary, 
participated.   The  children  were  placed  in  grades  three 
through  six  and  were  referred  to  the  writer  by  their  regular 
teachers.   The  sole  referral  criterion  was  that  the  child 
be  two  or  more  years  behind  in  his  reading  performance. 
Children  were  not  excluded  because  of  low  10  test  scores. 

Teachers 

Eleven  teachers  participated  in  this  study.   .All 
were  graduate  students  in  the  Department  of  Special  Education 
at  the  University  of  Florida.   Nine  were  Master's  candidates, 
one  an  Ed.D.  candidate,  and  one  a  Ph.D.  candidate.   All 
teachers  received  some  form  of  graduate  credit  for  their 
participation  in  the  study. 

Teaching  Situation 

The  children  in  the  siiudy  left  their  regular 
classes  at  a  predetermined  time  to  meet  in  a  group  with  the 
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special  teacher.   The  teaching  situations  varied  from  teacher 
to  teacher,  but  in  general  can  be  described  as  poor.   Lack 
of  space  necessitated  some  classes  meeting  in  cafeterias, 
gyms,  halls,  and  the  like.   Tn  all  cases,  however,  the  school 
administrators  in  each  school  made  every  effort  to  insure 
the  best  teaching  situation  available  in  the  specific 
circumstances . 

Curricular  Material 

The  Sullivan  Reading  Program  (1964)  was  selected 
for  use  in  this  study.   Consultation,  initially,  with 
educational  specialists  and,  later,  with  teachers  using  the 
program,  indicated  that  it  enjoyed  wide  acceptance  and  wag 
considered  educationally  sound. 

In  its  entirety,  the  Sullivan  Reading  Program 
is  not  all  programed  instruction.   Also  included  are 
storybooks,  Eilmstrips,  and  end-of-book  tests.  These 
materials  are  not  programed.   They  were  not  used  in  this 
study. 

The  programed  instruction  portion  of  the  Sullivan 
5£S^iH3_-EE2S£2!IL  ^-3   presented  in  three  series.   Series  I 
includes  Programmed  Reading  Booklets  1-7  (Grade  1) ,  Series 
II  includes  Programmed  Reading  Booklets  8-14  (Grade  2) , 
and  Series  III  includes  Programmed  Reading  Booklets  15-21 
(Grade  3) .   A  sample  of  the  program  from  Series  I  is 
presented  in  Appendix  A. 

Following  approximately  each  fifty  frames,  a 
Diagnostic  Test  is  scheduled.   The  Diagnostic  Test,  unlike 


1-3 
the  programed  text  proper,  has  no  answers  in  the  answer 
column.   The  purpose  of  the  Diagnostic  Test  is  to  give  the 
teacher  a  check  on  the  progress  of  the  student  by  presenting 
a  sample  of  the  content  presented  in  the  preceding  fifty 
frames.   A  sample  Diagnostic  Test  is  presented  in  Appendix  D. 

To  determine  the  individual  child's  proper 
starting  point  in  the  Programmed  Reader,  a  Placement  Test 
is  provided  with  the  Sullivan  Reading  Program.   The 
Placement  Test  is  similar  to  the  Diagnostic  Test  in  that 
it  has  no  answers.   Each  two  pages  in  the  Placement  Test 
summarize  the  content  in  one  Programmed  Reader.   The  child 
begins  in  the  booklet  indicated  by  his  first  error  in  the 
Placement  Test.   A  sample  Placement  Test  is  presented  in 
Appendix  C. 

Experimental  Design 

The  purpose  of  this  study  was  to  assess  the 
usefulness  of  free  operant  technology,  precision  teaching, 
in  the  analysis  of  programed  instruction  material,  specifically 
the  material  in  the  Programmed  Reader  of  the  Sullivan 
Re^ding_Program.   The  experimental  design  had  to  meet  two 
requirements : 

1)  it  had  to  use  the  Programmed  Reader  exactly 
as  indicated  in  the  Teacher's  Guide,  and 

2)  it  had  to  apply  free  operant  methods  to  the 
recording  and  modification  of  Programmed 
Reader  Performance. 

The  within-subject  design,  using  each  child  as  his  own 
control,  seemed  an  ideal  choice.   Following  Sicilian's  (1964) 
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suggestion,  the  experiment  was  conducted  in  phases  (the 
Before  Phase,  the  During  Phase  and  the  After  Phase)  with 
replications  across  children  and  teachers.   This  particular 
design  also  allowed  subsequent  between-subject  parametric 
analysis  of  many  of  the  questions.   This  is  schematically 
illustrated  in  FIGURE  I. 

In  the  first  phase,  the  Before  Phase,  the 
Placement  Test  was  administered  and  each  child  began  in 
the  Programmed  Reader  Booklet  indicated  by  his  Placement 
Test  score.   This  phase  lasted  approximately  eleven  days 
and  established  the  baselines  needed  to  evaluate  the  effects 
of  the  curriculum  and  the  teaching  procedure.   In  the  second 
phase,  the  During  Phase,  all  experimental  manipulations 
of  independent  variables  took  place.   This  phase  lasted 
approximately  eleven  days.   The  last  phase,  the  After 
Phase,  was  a  replication  of  the  Before  Phase  with  no 
experimental  manipulations  in  effect,  and  with  the  Placement 
Test  being  readministered  at  the  conclusion  of  approximately 
eleven  days. 

Experimental  design  applied  in  the  classroom 

The  teacher  in  the  classroom  intending  to  use 
the  Programmed  Reader  from  the  Sullivan  Reading  Program 
must  first  determine  in  which  Programmed  Reader,  (Booklet 
1-15)  each  child  is  to  begin.   The  Placement  Test  is  provided 
for  this  purpose.   If  performance  on  the  Placement  Test  is 
not  equivalent  to  performance  in  the  Programmed  Reader, 
then  the  Placement  Test  is  not  performing  its  stated  function 
with  respect  to  placing  the  pupil  in  hia  correct  beginning  booklet. 
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The  careful  teacher  will  want  to  know  if,  indeed, 
the  Placement  Test  score  does  approximate  the  level  of 
performance  of  the  booklet  indicated.   To  investigate  this 
question,  the  Placement  Test  was  readministered  on  the 
last  day  of  the  After  Phase.   If  the  Placement  Test  was 
correctly  indicating  Programmed  Reader  performance  level,  the 
final  administration  of  the  test  should  have  indicated  the 
book  in  which  the  child  was  last  performing. 

Skinner  points  out  (1954,  1958)  that  one  of  the 
central  goals  of  programed  instruction  is  to  maximize 
correct  responding  and  minimize  incorrect  responding. 
This  is  accomplished  by  carefully  constructing  each  programed 
frame  (antecedent  event)  in  the  program.   The  teacher  using 
fc^e  Sullivan  Reading  Program  naturally  wishes  to  know  whether 
the  Programmed  Reader  does,  in  fact,  differentiate  correct 
responding  from  incorrect  responding. 

To  examine  this  question,  correct  and  incorrect 
response  rates  on  the  Programmed  Reader  in  the  Before 
Phase  were  analyzed.   If  the  Programmed  Reader  differentiated 
correct  from  incorrect  responses,  then  a  difference  should 
exist  between  the  two  response  rates. 

The  most  recent  literature  in  programed  instruction 
(NSSE,  1967)  concerning  the  development  of  programs  suggests 
"in-program"  checks  with  frames  similar  to  those  in  the 
regular  program,  but  without  the  correct  solutions  available. 
The  Programmed  Reader  of  the  Sullivan  Reading  Program  has 
incorporated  this  principle  in  the  Diagnostic  Test,   The 
Diagnostic  Test,  according  to  the  Teacher's  Guide  to 
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Programmed  Reading  (1964),  is  not  to  be  graded,  but  instead 
to  be  used  as  a  guide  to  check  on  the  quality  of  the  students' 
work. 

The  careful  teacher  will  want  to  know  if  performance 
on  the  Diagnostic  Test  is  equivalent  to  performance  on  the 
Programmed  Reader  before  she  alters  her  supporting  curriculum 
on  the  basis  of  this  quality  check. 

To  evaluate  this  problem,  correct  and  incorrect 
performance  rates,  on  both  the  Diagnostic  Test  and  the 
Programmed  Reader,  collected  in  the  Before  Phase  were 
analyzed.   If  the  Diagnostic  Test  and  the  Programmed  Reader 
are  equivalent,  there  should  be  no  difference  between  the 
performance  rates  on  the  two  programs. 

The  teacher  who  has  taught  with  the  Programmed 
Readers  for  a  period  of  time  is  able  to  assess  the  performance 
of  her  students.   By  examining  each  child's  rate  correct 
and  rate  incorrect  as  plotted  on  his  six-cycle  serai-log  graph 
and  recorded  on  his  data  sheets  (Appendix  D) ,  she  can 
decide  on  the  best  tactic  to  maximize  his  performance 
accuracy.   That  is,  the  teacher  can  plan  how  to  increase 
the  difference  between  correct  response  rate  and  incorrect 
response  rate.   One  of  the  most  common  tactics  to  this  end 
is  the  arrangement,  the  presentation  or  withdrawal,  of  a 
subsequent  event  following  each  response  or  series  of 
responses  made  in  the  curricular  material.   This  arrangement 
can  be  made  with  the  intent  of  accelerating  correct  response 
rate  by  presenting  the  subsequent  event;  or  decelerating 
incorrect  response  rate  by  withdrawing  the  subsequent  event, 
or  both. 
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The  teacher  who  makes  such  an  arrangement  for  each 
child  will  want  to  know  precisely  what  effect  it  has  on 
the  child's  behavior.   In  addition,  she  will  want  to  know 
what  effect  the  arrangement  procedure,  itself,  has  on  the 
performance  of  her  class  as  a  whole. 

To  investigate  this  problem,  the  data  gathered 
in  the  Before  Phase,  where  no  experimental  conditions  were 
in  effect,  and  in  the  During  Phase,  where  arrangements 
were  in  effect,  were  compared.   If  the  subsequent  event 
arranged  to  follow  the  Programmed  Reader  correct  and/or 
incorrect  response  produced  an  effect  on  response  rate, 
this  wouldbe  seen  as  a  difference  between  response  rates 
in  the  Before  Phase  and  response  rates  in  the  During  Phase. 

The  classroom  teacher  is  only  too  aware  that  often 
pupils  are  not  motivated  to  perform  on  test  items.   It 
might  interest  her  to  know  whether,  on  the  Programmed 
Reader,  a  subsequent  event  presented  to  the  child  following 
each  errorless  Diagnostic  Test  will  alter  performance  rata 
on  the  Diagnostic  Test.   To  investigate  this  problem, 
correct  and  incorrect  performance  rates  on  the  Diagnostic 
Test  in  the  Before  Phase  and  in  the  During  Chase  were 
analyzed.   If  the  subsequent  event  following  each  errorless 
Diagnostic  Test  produced  an  effect  on  correct  and/or 
incorrect  response  rate,  this  would  be  seen  as  a  difference 
between  response  rates  in  the  Before  Phase  and  response 
rates  in  the  During  Phase. 

The  inquisitive  teacher,  when  she  presents  a 
subsequent  event  following  only  the  correct  Programmed 
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Reader  response  will  want  to  know  whether  the  correct 
response  rate  accelerates  or  not,  and  exactly  what  happens 
to  incorrect  response  rate  in  this  condition.   In  order  to 
answer  this  question,  incorrect  response  rates  on  the 
Programmed  Reader  in  the  Before  Phase  and  in  the  During 
Phase  were  compared.   If  the  subsequent  event  presented 
following  the  correct  Programmed  Reader  response  had  any 
effect  on  incorrect  Programmed  Reader  response  rate,  this 
would  be  seen  as  a  difference  between  incorrect  response 
rates  in  the  Before  Phase  and  incorrect  response  rates  in 
the  During  Phase. 

Similarly,  the  teacher  will  want  to  know  what 
the  effect  on  correct  response  rate  is  when  she  withdraws 
a  subsequent  event  following  the  incorrect  Programmed 
Reader  response.   The  same  procedure  was  repeated  for  the 
correct  response  rates  in  the  Before  Phase  and  in  the 
During  Phase  for  this  comparison.   If  the  subsequent 
event  withdrawn  following  an  incorrect  Programmed  Reader 
response  had  any  effect  on  correct  Programmed  Reader 
response  rate,  this  would  be  seen  as  a  difference  between 
correct  response  rates  in  the  Before  Phase  and  correct 
response  rates  in  the  During  Phase. 

Since  the  purpose  of  arranging  a  subsequent 
event  is  to  produce  a  lasting  change  in  the  magnitude  of 
the  difference  between  correct  and  incorrect  response  rates 
maintained  by  the  natural  consequences  of  superior  achievement, 
the  careful  teacher  will  want  to  know  precisely  what  the 
effect  of  the  removal  of  an  accelerating  or  decelerating 
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consequence  has  on  subsequent  performance.   The  problem 
was  investigated  by  comparing  correct  and  incorrect  Programmed 
Reader  response  rates  in  the  During  and  After  Phases.   If 
the  removal  of  a  consequating  condition  following  Programmed 
Reader  performance  produced  a  subsequent  change  in  Programmed 
Reader  response  rate,  that  change  would  be  observed  in  a 
difference  between  performance  rate  in  the  During  Phase  and 
performance  rate  in  the  After  Phase. 

Statement  of  the_Problem 
The  analyses  indicated  above  may  be  condensed  and 
summarized  in  the  following  eight  questions: 

1)  Is  the  Placement  Test  score  equivalent  to 
indicated  Programmed  Reader  book  number? 

2)  Is  correct  Programmed  Reader  response  rate 
different  from  incorrect  Programmed  Reader 
response  rate? 

3)  Is  performance  rate  on  the  Diagnostic  Test 
equivalent  to  performance  rate  on  the 
Programmed  Reader? 

4)  What  is  the  effect  of  the  arrangement  of 

a  subsequent  event  following  the  Programmed 
Reader  response  on  Programmed  Reader  response 
rate? 

5)  What  is  the  effect  of  a  subsequent  event 
following  each  errorless  Diagnostic  Test  on 
Diagnostic  Test  performance  rate? 

6)  What  is  the  effect  of  the  arrangement  of  a 
subsequent  event  following  each  Programmed  Reader 
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correct  response  on  Programmed  Reader  incorrect 
response  rate? 

7)  What  is  the  effect  of  the  arrangement  of  a 
subsequent  event  following  a  Programmed 
Reader  incorrect  response  on  Programmed 
Reader  correct  response  rate? 

8)  What  is  the  effect  of  the  removal  of  the 
accelerating  and  decelerating  consequences 
following  the  Programmed  Reader  response  on 
Programmed  Reader  performance  rate? 


Chapter  II 


RESULTS 


Placement  Test  Accuracy 

The  Sullivan  Reading  Program  is  organized  so  that 
a  child  may  begin  at  any  performance  level  from  grades  1 
through  4.   The  Programmed  Reader  booklet  in  which  he 
begins  is  determined  by  his  performance  on  the  Placement 
Test.   The  child  begins  in  the  booklet  indicated  by  the 
location  of  his  first  error  in  the  Placement  Test. 

To  determine  the  accuracy  of  the  Placement  Test, 
the  scores  on  pre-Placement  Tests  administered  in  the 
Before  Phase,  the  scores  on  post-Placement  Tests  administered 
at  the  conclusion  of  the  After  Phase,  and  the  booklet 
numbers  of  the  last  Programmed  Readers  completed  in  the 
After  Phase,  were  compared  by  means  of  a  simple  analysis 
of  variance.   The  results,  presented  in  TABLE  I,  suggest 
that  a  reliable  difference  existed  among  the  three  measures. 
A  t  test  between  the  last  Programmed  Reader  booklet 
completed  and  the  post-Placement  Test  yielded  a  t  of  12.66, 
p   .01.   Strong  support  is  lent  to  the  conclusion  that 
the  Placement  Test  was  not  a  reliable  estimate  of  Programmed 
Reader  performance. 
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TABLE  I 

A  Simple  /.nalysis  of  Variance  for  Differences  Betwnon 

Pre-Placement  Test  Scores,  last  Completed  Progranmed 

Reader  Booklet  Number,  and  Post-Placcnent  Test  Scores 


Source  of  Variation         df         Variance 

Estimate 


Between  Groups  2  278.43        45.2* 

Within  Groups  56  6.27 

Total               58           15.55 
,_ — ^__ DT_ 
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The  Difference  Between  Correct  and  Incorrect  Programmed  Reader 
Performance  Rates 

In  setting  up  guidelines  for  writing  programs, 

Skinner  (1954)  pointed  out  that  the  well-constructed 

program  should  maximize  correct  responding  while  permitting 

a  minimum  number  of  error  responses.   The  question  of  this 

difference  was  examined  in  the  Programmed  Reader  by  treating 

the  correct  and  incorrect  Programmed  Reader  response  rates 

collected  in  the  Before  Phase  with  the  Lindquist  Type  I 

Analysis  of  Variance  (Lindquist,  1953) .   TABLE  II  indicates 

that  correct  and  incorrect  response  rates  on  the  Programmed 

Reader  were  well  differentiated.   TABLE  III  and  FIGURE  II 

display  the  direction  and  magnitude  of  this  difference  for 

the  entire  population  and  for  one  typical  child,  respectively. 

The  Difference  Between  Programmed  Reader  Performance  Rates 
and  DiagriOstTc~Test  Performance  Rates 

In  the  Programmed  Reader  booklets,  a  Diagnostic  Test  is 
scheduled  approximately  each  50  frames.   The  Diagnostic 
Test  is  intended  to  be  a  quick  check  on  the  accuracy  of 
pupil  performance.   If  this  check  is  to  be  meaningful, 
the  correct  and  incorrect  Diagnostic  Test  performance  rates 
must  be  approximately  equal  to  the  correct  and  incorrect 
performance  on  the  Programmed  Reader  itself. 

The  extent  of  the  difference  was  analyzed  with 
the  Lindquist  Type  VI  Analysis  of  Variance.   This  analysis, 
presented  in  TABLE  IV,  considered  class  units,  in  addition 
to  both  correct  and  incorrect  response  rates  on  the  Diagnostic 
Test  and  the  Programmed  Reader,   It  is  clear  that  a  reliable 
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TABLE  III 

The  Direction  and  Magnitude  of  Differences  Between  Correct 
and  Incorrect  Programmed  Reader  Response  Rates 


Direction  of  Magnitude  of 

Child  Change  Change 

Allen  +  3.20 

Ballard  +  12.60 

Carver  +  6.35 

Hathway  +  5,15 

Williams  +  9.70 

Langston  +  6,3  5 

Hines  +  3.50 

Willford  +  2.90 

Young  +  5,45 

Godbolt  +  2,4  0 

Nattiel  +  3.70 

Ford  +  2,70 

Brown,  M.  +  13.75 

Wright  +  6.95 

Jones  +  6.95 

Jeff coat  +  4.50 

Fogarty  +  6.85 

Howell  +  14.83 

Jackson  y  7,20 

Wimms  +  5.75 

Thomas  +  4.85 

Klickly  +  4.55 

Lesene  +  4.50 

Webb  +  4.40 

Morris,  p.  +  4.06 

Morris,  B.  +  9.67 

Johnson  +  4.35 

Hardwic  +  4.15 

Cray  +  5.71 

Hayes  +  7.29 

Camps  +  6.02 

Kelly  +  5.90 

Railey  +  1.90 

Sperring  +  7.15 

Taylor  +  3.50 

Bruce  +  4.65 

Hague  +  9.10 

Howell,  D.  +  4.90 

Beals  +  4.80 

Bishop  +  3.70 

Somese  +  3.80 

Lee  +  3.40 


TABLE  III  Continued 
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Child 


Direction  of 
Change 


Magnitude  of 
Change 


Pattison 

Ross 

Brown 

James 

Stewart 

Walker 

Berry 

Haile 

Alexander 

Burke 

Brown,  K. 

Davis 

Smith,  J. 

Bass 

Fogarty 

Howard 

Shaupe 


+ 
+ 

+ 
+ 

+ 
+ 
+ 
+ 
+ 
+ 

-!- 
+ 
+ 
+ 

+ 
+ 


7  0 
90 
13 
68 
68 
93 
62 
20 
55 
30 
05 
20 
05 
95 
90 
90 
48 
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30 
difference  did  exist  between  performance  rates  on  the 
Diagnostic  Test  and  on  the  Programmed  Reader,  with  the  latter 
having  the  higher  rates.   In  addition,  differences  were 
observed  between  correct  and  incorrect  response  rates 
(a  result  consistant  with  TABLE  II)  and  in  performance 
rates  among  class  units. 

A  graphic  display  of  the  means  in  the  interaction 
terras  revealed  that:   1)  the  B  x  C  interaction  could  be 
attributed  to  the  differences  observed  between  the  mean 
incorrect  response  rate  on  the  Diagnostic  Test  and  the  mean 
incorrect  response  rate  on  the  Programmed  Reader,  and  2) 
the  A  x  B  interaction  could  be  attributed  to  a  pooling  of 
four  distinct  class  units  on  correct  response  rates  for 
both  the  Diagnostic  Test  and  Programmed  Reader. 

The  Effects  of  Arranging  a  Subsequent  Event  Following 
P r o qrammed  Reader  Responses 

The  precision  teacher  who  wishes  to  change  the 

rate  of  correct  and/or  incorrect  responding  often  achieves 

this  end  by  arranging  a  subsequent  event  (e.g.,  penny, 

star,  M&M,  etc.)  to  follow  the  desired  response.   This 

procedure  of  presenting  or  withdrawing  subsequent  events 

may  be  used  to  either  accelerate  or  decelerate  the  response 

rate.   The  first  possible  procedure,  presenting  a  subsequent 

event  following  correct  Programmed  Reader  responses,  while 

ignoring  incorrect  Programmed  Reader  responses,  was 

examined  by  treating  the  correct  Programmed  Reader  response 

rates  collected  in  the  Before  and  During  Phases  of  the  experiment 

with  the  Wilcoxon  Signed  Rank  Test  (Wilcoxon  and  Wilcox,  1964). 
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The  result,  presented  in  TABLE  V,  suggests  that  a  reliable 
difference  was  present  between  the  performance  rates  in  the 
two  phases.   An  examination  of  column  "d"  shows  that  most 
of  the  response  rates  were  accelerated  in  the  During  Phase. 
It  should  be  noted  that  the  magnitude  of  each  of  the  seven 
accelerations  was  far  greater  than  the  magnitude  of  each 
of  the  two  decelerations.  A   graph  of  this  acceleration  is 
presented  in  FIGURE  III. 

The  second  possible  procedure,  withdrawing  a 
subsequent  event  after  incorrect  Programmed  Header  responses, 
while  ignoring  correct  Programmed  Reader  responses,  was  also 
treated  with  the  Wilcoxon  Signed  Rank  Test.   The  result  as 
presented  in  TABLE  VI,  indicates  a  reliable  difference 
between  the  performance  rates  in  the  two  phases,  hn 
examination  of  column  "d"  shows  that  the  response  rates 
here  were  decelerated  in  the  During  Phase.   Once  again, 
the  magnitude  of  the  changes  was  largest  in  the  expected 
direction  of  the  change,  deceleration  in  this  case.   A 
graph  of  this  deceleration  is  presented  in  FIGURE  IV. 

The  third  possible  procedure,  the  combination  of 
the  first  two  arrangements,  was  also  treated  with  the 
Wilcoxon  statistic.   TABLES  VII  and  VHI  present  the 
results  of  the  analyses.   It  should  be  noted  the  results  of 
this  combination  of  arrangements  are  strikingly  similar  to 
each  of  the  arrangements  alone  for  their  respective 
responses.   FIGURE  V  displays  this  simultaneous  acceleration 
and  deceleration. 


32 


01 

0 

n 

1H  13    1) 

0! 

c  -P 

K 

aS   as 

a, 

c 

m 

<D 

o>  <d 

HI 

JJ    M 

s 

nj   C 

+1 

K   0 

0 

a 

m 

a)  w 

to 

c  K 

0) 

0 

0 

D  u 

c 

m  a) 

01 

a)  •a 

H 

K    is 

ID 

m 

<H 

l-l  K 

in 

a) 

•h  -a  >o 

> 

D 

H!   01 
01  s 

w 

^  «  • : 

iA 

0 

g 

wo  n 
o>  tn 

Ch 

-u 

g   0 

m 

1  n 

0 

r3  Cu 

fr< 

.* 

0  u 

C 

U   01 

as 

di   1-1 

« 

u 

■P  o 

■0 

o  u 

0) 

01 

c 

M   0) 

tjl  1-1   w 

•H 

0  <d 

to 

0  S. 

c 

0) 

0 

m  C 

X 

as  a 

0  -G  -H 

u 

p.  1-1 

H 

3 

■H 

o 

m 

111 

•H 

u 

01 
0) 

en 

4h 

i-l 

U 

0 

c 

ss 

0) 

M 

+> 

0 

C 

•H 

rr 

f.l 

s 

0) 

PJ 

u 
fa 

y,  a 

K 


m 

c 

0) 

■H 

in 

n 

m 

3  .c 

a 

A 

0) 

M 

0) 

O 

n 

>H 

in 

A 

J3 
P< 

^•u3for-m<T\cotH(N 


COOVDfNCOCNC^Om 


n  n  rH  n  m  «*  ot 

+  +  +  +  +  +  +    l    I 


coa\Ha>rainci^in 


koi^r^cor'OH'i'fo 


^•^inm^wr-^n 


c 

•p 

0 

M 

u 

w  a 

c 

ifl 

tfl    d) 

>i  01 

yi  -H  o 

'£ 

01  M 

n  (.1 

0J  -P  .G 

0 

<S) 

B  H 

H   R 

>l4J   O 

u 

4J 

f3    (T3 

<u  ra 

nj   a)  -H 

m 

us 

hi  B 

H  U 

a  P<  « 

II 

Eh 


o 
V 


33 


tj> 

c 

•H 

S 

0   4J 

H    0 

rH     0) 

0    U 

t-i   u 

0 

■P  o 

c 

G   C    0 

>   0  -P 

0         fl) 

Ifi    >H 

-P  o 

emu 

o  C  m 

3   0    H 

truo 

H 

0)  in  p. 

H 

m  <D  w 

H 

ja  u  o 

3         l-l 

a 

D 

ra  >a  a) 

U 

nj  t! 

H 

CP  o  ra 

fc. 

B  «  a 

■H          « 

+J  "0 

C»JB 

(!)    g   ff 

6   c  £ 

0)   fy    E 

M    M   3 

P.  Oi  ^ 

0  Cn 

"HMO 

0    fc    >H 

p. 

■4J  4J 

u  0 

0)    0 

M-l   H 

t-l    H 

0   0 

o 

<D 

.c 

e< 

'(0, 


t4-t 


V 


G 


V 


% 


jjj'l 

iil'T   ill 


Ujiii 


LUi 

I!:!' 


.11! 


\  ■  I  ■    :  '•     i 
iij_uj:.  -UJ.J.- 


u 


]-;Z 


8§ 


lifjliiE 


i-  i: 


VI' 
•iitr 


HTj: 

!'■!■"! 


1  ! 

.  .■  r 


ii  i  -. 


-1" 


"Oils 

:  I--::- 

!  m 

m 


Q  «       -  «i       -  g       53 

3lnNIW  83d  S.INJH3AOW 


o 


o 


0 


8 


0 
01 


0 

8 

o 

10 


0 


o 
-a 


34 


TABLE  VI 

A  Wilcoxcn  Signed  Rank  Teat  for  Differences  Between 

Before  Phase  Incorrect  Programmed  Reader  Response  Rates 

and  During  Phase  Incorrect  Programmed  Reader  Response  Rates 


Before 

During 

Rank  of 

Rank 

of  D 

Child 

Phase 

Phase 

d 

D 

wi  th 

Less 

Frequent  Sign 

Will ford 

.75 

.40 

-.35 

16 

Ballard 

.39 

.05 

-.34 

15 

Williams 

.24 

.08 

-.16 

10 

Brown 

.29 

.24 

-.05 

4 

Howell 

.23 

.46 

+  .23 

1  3 

13 

Kelly 

.54 

.03 

-.51 

17 

Davis 

.10 

.12 

+  .02 

1.5 

1.5 

Brcwn 

.OS 

.02 

-.06 

5.5 

Burke 

.22 

.07 

-.15 

9 

Alexander 

.09 

.15 

+  .06 

5.5 

5.5 

Haile 

.36 

.14 

-.22 

11.5 

Morris,  B. 

.42 

.10 

-.32 

14 

Morris,  P. 

1.06 

.20 

-.86 

18 

Jackson 

21 

.13 

-.08 

7 

Klickly 

.25 

.24 

-.01 

1 

Lesene 

.42 

.20 

-.22 

11.5 

Thomas 

.69 

.59 

-.10 

8 

Wimms 

.09 

.11 

+  .02 

1.5 

T  i 

1.5 
=  21.5* 

*P  <  .005 
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TABLE  VII 

A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  Before 

Phase  Correct  Programmed  Reader  Response  Rates  and  During 

Phase  Correct  Programmed  Reader  Response  Rates 


Before 

During 

Rank  of 

Rank  of  D 

Child 

Phase 

Phase 

d 

D 

with  Less 
Freauent  Sign 

Shaupe 

3.74 

3.08 

-.66 

9 

9 

Smith,  J. 

3.01 

2.31 

-.70 

10 

10 

Bass 

3.45 

5.94 

+2.49 

25 

Fog arty 

3.20 

5.23 

+2.0  3 

22 

Howard 

6.24 

4.47 

-1.77 

21 

21 

Ford 

3.35 

3.58 

+  .23 

4 

Godbolt 

3.00 

3.36 

+  .36 

7 

Hines 

3.91 

5.23 

+1.32 

19 

Nattiel 

3.65 

4.71 

+1.06 

16.5 

Young 

5.25 

5.37 

+  .62 

8 

Allen 

2.41 

1.67 

-.74 

12 

12 

Carver 

6.51 

6.17 

-.34 

6 

6 

Kathway 

5.70 

6.73 

+1.03 

13.5 

Langs  ton 

6.52 

7.55 

+1.03 

13.5 

Jones 

6.78 

7.49 

+  .71 

11 

Wright 

7.35 

11.10 

+3.75 

30 

Jef fcoat 

4.86 

7.63 

+2.27 

29 

Fogarty 

7.21 

12.57 

+5.36 

31 

Seals 

5.26 

4.19 

-1.07 

18 

18 

Lee 

3.78 

3.97 

+  .19 

3 

Ross 

6.23 

6.24 

+  .01 

1 

Somese 

4.21 

6.86 

+2.65 

26 

Cray 

6.77 

4.11 

-2.66 

27 

27 

Hardwic 

5.06 

3.12 

-1.06 

16.5 

Johnson 

6.12 

3.39 

-2.73 

28 

28 

Webb 

5.27 

4.22 

-1.05 

15 

15 

Raj.ley 

2.48 

2.43 

-.05 

2 

2 

Sperring 

7.50 

9.20 

+1.70 

20 

Taylor 

4.05 

6.31 

+2.26 

23 

Bruce 

4.78 

7.25 

+2.74 

24 

Hague 

9.10 

15.20 

+6.10 

32 

Howell 

5.64 

5.33 

-.22 

5 

5 

T  =  153* 
z  =  2.07 

*P  <  .02 
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TABLE  VIII 


A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  Before 

Phase  Incorrect  Programmed  Reader  Response  Rates  and  During 

Phase  Incorrect  Programmed  Reader  Response  Rates 


Before    During  Ra"nk  of     Rank"  of  D 

Child    Phase     Phase      d        D        with  Less 


Shaupe 

.27 

Smith 

.08 

Bass 

.20 

Fog arty 

.18 

Howard 

.16 

Ford 

.06 

Godbolt 

.54 

Hines 

.34 

Nattiel 

.21 

Young 

.20 

Allen 

.45 

Carver 

.45 

Hathway 

.36 

Lang s ton 

.28 

James 

.56 

Wright 

.37 

Jeff coat 

.25 

Fogarty 

.15 

Beal 

.43 

Lee 

.30 

Ross 

.41 

Somese 

.16 

Cray 

.32 

Hardwic 

.50 

Johnson 

.56 

Webb 

.92 

Railey 

.44 

Sperring 

.40 

Taylor 

.19 

Bruce 

.12 

Hague 

.18 

Howell 

.09 

.1.1 

-.16 

14.5 

.04 

-.04 

1.5 

.11 

-.09 

5 

,07 

-.11 

9.5 

,28 

+  .12 

11 

.42 

+  .36 

27.5 

.33 

-.21 

19 

,13 

-.21 

19 

.01 

-.30 

24 

,10 

-.10 

7 

,11 

-.34 

25 

,08 

-.37 

2  9 

,21 

-.15 

13 

,12 

-.16 

14.5 

,16 

+  .04 

1.5 

16 

-.21 

19 

15 

-.10 

7 

,15 

-0 

21 

-.22 

21 

15 

-.17 

16 

16 

-.25 

22 

08 

-.08 

4 

18 

-.14 

12 

14 

-.36 

27.5 

21 

-.35 

26 

20 

-.72 

31 

04 

-.40 

30 

14 

-.26 

2  3 

09 

-.10 

7 

01 

-.11 

9.5 

00 

-.18 

17 

14 

+  .05 

3 

■Frequent  Sign 


11 
27.5 


1.5 


T  ■  43* 
Z  =  4.01 


"*P  <  .0000T 
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The  Effects  of  the  Arrangement  of  a  Subsequent  Event 
Following  Each  Errorless  "Diagnostic  Test 

It  has  just  been  demonstrated  that  the  presentation 

and  withdrawal  of  a  subsequent  event  following  Programmed 

Reader  performance  reliably  altered  the  rate  of  that  performance. 

What,  then,  is  the  effect  of  presenting  a  subsequent  event 

following  each  errorless  Diagnostic  Test?   To  answer  this 

question,  a  Lindquist  Type  VI  Analysis  of  Variance  was 

performed  on  all  Diagnostic  Test  response  rates,  correct 

and  incorrect.   The  data  were  analyzed  by  phases,  Before 

Phase  and  During  Phase;  with  a  subordinate  analysis  by 

class  units.   It  can  be  seen  in  TABLE  IX  that,  although 

the  main  effect  for  class  units  and  the  main  effect  for 

correct  and  incorrect  response  rates  were  significant,  the 

Before  and  During  Phase  main  effect  was  not  significant. 

This  result  is  clearly  evident  in  the  lack  of  uniformity 

in  the  direction  and  magnitude  of  changes  presented  in 

TABLE  X. 

The  Effect  of  Presenting  a  Subsequent  Event  Following 
Correct  Programmed  Reader  Responses  on  Incorrect  Programmed 
Reader  Response  Rate 

It  was  previously  demonstrated  in  TABLE  V  that  the 

presentation  of  a  subsequent  event  following  the  correct 

Programmed  Reader  response  accelerated  that  response.   In 

that  condition,  the  incorrect  Programmed  Reader  response 

was  ignored.   To  investigate  the  effect  of  the  presented 

subsequent  on  the  previously  ignored  incorrect  response  rate, 

a  Wilcoxon  Signed  Rank  Test  was  performed  on  the  incorrect 

Programmed  Reader  response  rates  observed  in  the  Before  and 


40 


x 


a 

< 


0) 


o 

a 


a 
O  oi 

c  <u 

CJ  -P    0) 

>-i  m  <u 

m      id 

•M  oi 
a  c 
o 
n  & 
o  to 

W   01 

<u      o 

O  -f  K 

C  01 

nj  0  jj 

•H  Eh  W 

^  0) 

<fl  O  Eh 

>  -rt 

■P  o 

»4    Ji-rl 

0  O  4J 

oi  tr  O 
•rt  18  a 

oi  -h  tj> 

C    01 

K  id  a) 

A   01 

h  e.  id 

>  fl 

0)  (i 

a)  ti 

a  o  oi 

>1<W   c 

E-i   0)  -H 

■(J        3 

01  G  Q 

•h  <u 

3  <y  t3 
t?  3  c 
td  -P  id 
a  a) 

•H   B 


«  c 

o  o 

•H 

0)  -P 

0  ID 

3  M 

o  us 

w  > 


t*  c-  oi 

-D  o  «J 

<r  cm  cn 


4-t  CO  O  00 

■O   /1HV 


c  01  — 

<d  a) 
<u  01  u 
s  oi  o 

din 
<u  r 


MOM 


*   *   *   * 
r*  i£>  in  r~  n 

Mi\cin  ^ 

fo  cr>  co  in  r- 


m  cTi  in  *r  a*. 

o  oi  n  >  o 


■*  vo  r^  in  in 

IN  IN 


ro  <y\  vp  own 

o  aurq'Oi 


rr  mj  n  i^  o 
cm  r~  in  in 


.—)  H  o  o  o 

rH  rH  ,H 


<?  cri  vc  r-» 
r.  o  o  M3 


<fOHM 
<3»  CO 


=3*  03  oo  cc 

<?  *r  *a»  •* 


•P 

c  o 

H    O 

a  ^ 
-u  n 
o 


U 


a  i  x 

Mh  cnuun 

0    0  -H 

«   CJ   3 


SUH«Q<<CQ< 


^   U 
0   0 

u  u 


en  in 

■  i-H 

I    • 

en  in 
H  in 

CO  CM 


o 

:i 
0 

a 

(X, 

o 

•H 

in  4J 
n  u 
cm  ai 
n  , 
jj 
•a  o 

•P  O 

o 

Eh 


41 


TABLE  X 


The  Direction  and  Magnitude  of  Differences  Between  Before  Phase 
Correct  and  Incorrect  Programmed  Reader  Responses  and  During 
Phase  Correct  and  Incorrect  Programmed  Reader  Responses 


Correct 

Incorrect 

Direction  of 

Magnitude  of 

Direction  of 

Magnitude  of 

Child 

Change 

Change 

Change 

Change 

Allen 

_ 

.40 

_ 

.15 

Ballard 

- 

4.00 

- 

1.00 

Carver 

00 

- 

.92 

Hathway 

- 

3.00 

00 

Williams 

+ 

1.00 

- 

.71 

Langs ton 

+ 

2.80 

+ 

.23 

Hines 

+ 

5.40 

- 

.60 

Willford 

+ 

2.70 

- 

.72 

Young 

+ 

5.40 

- 

.30 

Godbolt 

+ 

.60 

- 

.34 

Nattiel 

+ 

3.00 

- 

.60 

Ford 

+ 

2.40 

- 

.45 

Brown,  M. 

- 

1.70 

+ 

.11 

Wright 

+ 

1.00 

- 

.58 

Jones 

- 

.70 

- 

.34 

Jeff coat 

- 

2.70 

+ 

.32 

Fogarty 

+ 

2.00 

- 

.33 

Howell 

00 

+ 

.35 

Jackson 

+ 

1.50 

+ 

.50 

Wimms 

+ 

2.50 

+ 

.80 

Thomas 

+ 

.90 

- 

.33 

Kickly 

+ 

2.20 

- 

.33 

Lesene 

+ 

.90 

- 

.52 

Webb 

- 

.60 

+ 

.36 

Morris,  P. 

- 

2.40 

- 

1.20 

Morris,  B. 

+ 

1.20 

- 

.90 

Johnson 

+ 

4.20 

+ 

.60 

Hardwic 

- 

1.50 

- 

1.20 

Cray 

+ 

.90 

f 

.30 

Hayes 

+ 

3.00 

■r 

.39 

Camps 

+ 

7.30 

- 

.09 

Kelly 

+ 

1.00 

- 

1.50 

Railey 

+ 

4.30 

- 

.30 

Sperring 

+ 

3.40 

+ 

.15 

Taylor 

+ 

3.10 

- 

.01 

Bruce 

+ 

1.20 

+ 

.44 

Hague 

+ 

2.00 

- 

.22 

Howell,  D, 

+ 

1.40 

- 

.57 

Beals 

+ 

.30 

+ 

.06 

Bishop 

+ 

1.80 

- 

.42 

Somese 

-. 

2.30 

+ 

.27 

Lee 

+ 

.90 

+ 

.54 
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TABLE  X  Continued 


Child 


Correct 
Direction  or"  Magnitude  of 
Change        Change 


Incorrect 
Direction  ~ol     Magnitude  of 
Change        Change 


Pattison 

+ 

Ross 

Brown 

+ 

James 

Stewart 

+ 

Berry 

- 

Haile 

- 

Alexander 

- 

Burke 

Brown,  K. 

+ 

Davis 

Smith,  J. 

+ 

Bass 

+ 

Fogarty 

- 

Howard 

- 

Shaupe 

- 

1.30 

00 

4.50 

CO 

1.00 

.20 

.80 

3.50 

.40 

00 

.40 

00 

.20 

.21 

.30 

2.60 

2.00 


+ 
+ 

+ 


.54 
.49 
.63 
00 
.25 

1.79 

1.05 

00 

,33 

.50 

.94 

3.50 

.15 

00 

.34 

3.45 
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During  Phases.   TABLE  XI  shows  that  incorrect  Programmed 
Reader  response  rates  were  reliably  decelerated  when  an 
arrangement  was  made  to  follow  correct  Programmed  Reader 
responses.   An  examination  of  column  "d"  reveals  that  in 
only  two  cases  was  an  acceleration  observed,  and  these 
cases  showed  the  least  changes  in  magnitude.   FIGURE  VI 
displays  this  deceleration  as  observed  in  one  typical  child's 
performance. 

The  Effect  of  Withdrawing  a  Subsequent  Event  Following 
Incorrect  Programmed  Reader  "Responses  on  Correct  'Programmed 
Reader  Response  Rate 

TABLE  VI  demonstrated  that  withdrawal  of  a  subsequent 

event  fo."  lowing  the  incorrect  Programmed  Reader  response 

decelerated  incorrect  response  rate.   In  that  condition,  the 

correct  Programmed  Reader  response  was  ignored.   To 

investigate  the  effect  of  the  presented  subsequent  event  on 

the  previously  igrored  correct  response  rate,  a  Wilcoxon 

Signed  Rank  Test  was  performed  on  the  correct  Programmed 

Reader  response  rates  observed  in  the  Before  and  During 

Phases.   TABLE  XII  shows  that  correct  Programmed  Reader 

responses  were  not  reliably  affected  by  the  withdrawal 

of  a  subsequent  event  following  the  incorrect  Programmed 

Reader  response.   An  examination  of  column  "d"  in  TABLE  XII 

reveals  that,  although  the  direction  of  changes  was 

predominately  accelerating,  the  magnitude  of  changes 

observed  in  the  four  cases  of  deceleration  was  sufficiently 

large  to  counteract  the  directional  predomenance  of  the 

data  and  yield  a  non-significant  result. 
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TABLE  XI 

A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  Before 

Phase  Incorrect  Programmed  Reader  Response  Rates  and 

During  Phase  Incorrect  Programmed  Reader  Response  Rates 


Before 

During 

Rank  of 

Rank  with 

Child 

Phase 

Phase 

d 

D 

Less 
Frequent 
Sign 

Brown 

.38 

.28 

-.10 

3.5 

Stewart 

.20 

.07 

-.13 

5 

James 

.12 

.10 

+  .04 

2 

2 

Walker 

.39 

.20 

-.19 

6 

Berry 

.51 

.25 

-.26 

8 

Camps 

.20 

.10 

-.10 

3.5 

Hayes 

.29 

.08 

-.21 

7 

Bishop 

.18 

.19 

+  .01 

1 

1 

Pattison 

.06 

.06 

0 

T  =  3* 

*P  <  .OS 

4 '5 


u 

o 
0  u 


8§     3S     2 
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TABLE  XII 

A  Wilooxon  Signed  Rank  Test  for  Differences  Between 

Before  Phase  Correct  Programmed  Reader  Response  Rates 

and  During  Phase  Correct  Programmed  Reader  Response  Rates 


Child 


Before 
Phase 


During 
Phase 


Rank  of 
D 


Rank  with 
Less 

Frequent 
Sign 


* 


Willford 

Ballard 

Williams 

Brown 

Howell 

Kelly 

Davis 

Brown 

Burke 

Alexander 

Haile 

Morris,  B. 

Morris,  P. 

Jackson 

Klickly 

Lesene 

Thomas 

Wimms 


3.77 
13.41 

9.82 
14.05 
15.15 

6.22 

2.25 


98 
49 


2.65 


,82 
,89 
,51 
,86 
,81 


4.97 
5.90 
6.83 


4.32 

+  .55 

12.79 

-.62 

9.87 

+  .05 

17.76 

+3.71 

15.50 

+  .35 

4.43 

-1.79 

3.20 

+  .84 

5.63 

+1.65 

2.78 

+  .29 

3.15 

+  .50 

2.91 

+  .09 

7.50 

-2.39 

5.66 

+  .15 

10.48 

+3.62 

4.44 

-.37 

6.95 

+1.98 

5.98 

+  .08 

11.07 

+4.24 

10 

11 

1 

7.5 

6 
14 
.12 
13 

5 

9 

3 
IS 

4 
17 

7.5 
15 

2 
18 


11 


14 


16 


7.5 


48.5* 


*not  significant 
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The  Effects  of  Terminating  Arrangements  on  Subsequent 
Programmed  Reader  Performance 

JJormally  it  is  not  practical  to  maintain  a  classroom 
under  continual  synthetic  consequation.   The  effects  of 
presenting  subsequent  events  following  Programmed  Reader 
responses  have  been  previously  examined.   What  is  the 
effect  of  removing  the  arrangement  entirely? 

In  the  first  condition,  where  the  subsequent  event 
was  presented  following  the  correct  Programmed  Reader  response, 
the  data  collected  in  the  During  and  After  Phases  were 
treated  with  the  Wilcoxon  Signed  Rank  Test.   TABLE  XIII 
indicates  that  the  already  accelerating  correct  Programmed 
Reader  ri  sponse  rates  continued  accelerating.   An  examination 
of  column  "d"  reveals  that  in  no  case  was  a  deceleration 
observed.   This  effect  is  shown  for  a  typical  child  in 
FIGURE  VII. 

In  the  second  condition,  where  the  subsequent 
event  was  withdrawn  following  the  incorrect  Programmed 
Reader  response ,  the  incorrect  response  rates  for  these 
children  that  were  collected  during  the  During  and  After 
Phases  were  treated  with  the  Wilcoxon  statistic.   No  change 
in  incorrect  response  rate  is  observed  in  TABLE  XIV.   An 
examination  of  column  "d"  reveals  no  consistancy  in 
either  the  direction  or  magnitude  of  the  changes.   FIGURE 
VIII  demonstrates  this  maintenence  of  response  rate. 

The  third  condition,  where  both  of  the  above 
arrangements  were  in  effect,  was  also  treated  with  by 
applying  Wilcoxon  Signed  Rank  Test  to  the  performance  rates 
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TABLE  XIII 

A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  During 
Phase  Correct  Programmed  Reader  Response  Rates  and  After 
Phase  Correct  Programmed  Reader  Response  Rates 


Child 


During 
Phase 


After 
Phase 


Rank  of 

Rank  with 

D 

Less 

Frequent 

Sign 

Brown 

6.83 

11.17 

+4 .  34 

Stewart 

7.97 

9.97 

+2.02 

James 

7.16 

10.22 

+3.06 

Walker 

8.92 

12.10 

+3.18 

Berry 

7.82 

8.68 

+  .86 

Camps 

10.56 

11.87 

+1.31 

Hayes 

11.93 

13.65 

+1.72 

Bishop 

3.54 

7.22 

+  3.68 

Pattison 

4.45 

8.23 

+3.78 

*P  <  .01 

0* 
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TABLE  XIV 


A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  During 

Phase  Incorrect  Programmed  Reader  Response  Rates  and 

After  Phase  Incorrect  Programmed  Reader  Response  Pates 


During 

After 

Child 

Phase 

Phase 

Will ford 

.40 

.17 

Ballard 

.05 

.03 

Williams 

.08 

.18 

Brown 

.24 

1.46 

Howell 

.46 

.47 

Kelly 

.03 

.07 

□avis 

.12 

.07 

Brown 

.02 

.01 

Burke 

.07 

.15 

Alexander 

.15 

.02 

Haile 

,14 

.21 

Morris,  B. 

.10 

.21 

Morris,  P. 

.20 

.,61 

Jackson 

.13 

.45 

Klickly 

.24 

.17 

Lesene 

.20 

.52 

Thomas 

.59 

.66 

Wimns 

.11 

,23 

Rank  of     Rank  with 
D  Less 

Frequent 
Sign 


-.23  14  14 

-.02  3  3 

+.10  10 

+1.22  18 

+.01  1.5 

+  .04  4 

-.05  5  5 

-.01  1.5  1.5 

+  .08  9 

-.13  13  13 

+  .07  7 

+.11  11 

+  .41  17 


+.32  15. 

Klickly       .24        .17    -.07  7 

Lesene        .20        .52    +.32  15. 

+  .07  7 

+.12  12 


=  43.5' 


•not  significant 
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for  both  correct  and  incorrect  Prograiamed  Reader  responses 
in  the  During  and  After  Phases.   The  results  of  these 
analyses  are  presented  in  TABLES  XV  and  XVI.  Note  the 
striking  similarity  of  these  results  to  those  observed  in 
TABLES  XIII  and  XIV.   In  both  cases  an  acceleration  was  observed 
for  correct  response  rates  and  no  effect  was  observed  for 
incorrect  responses.   FIGURE  IX  demonstrates  this  simultaneous 
acceleration  and  maintenence. 

Summary  of  Res u Its 

The  results  of  this  study  may  be  summarized  as 


follows: 


1)  the  Placement  Test  is  not  an  accurate  index 
of  Programmed  Reader  performance 

2)  correct  and  incorrect  Programmed  Reader 
response  rates  are  reliably  different 

3)  the  presentation  of  a  subsequent  event 
following  the  correct  Programmed  Reader 
response  reliably  accelerates  that  response 

4)  the  withdrawal  of  a  subsequent  event  following 
the  incorrect  Programmed  Reader  response 
reliably  decelerates  that  response 

5)  the  presentation  of  a  subsequent  event  following 
each  errorless  Diagnostic  Test  does  not 
reliably  effect  Diagnostic  Test  performance 
rate 

6)  the  presentation  of  a  subsequent  event 
following  the  correct  Progrannned  Reader 
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TABLE  XV 


A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  During 
Phase  Correct  Programmed  Reader  Response  Rates  and  After 
Phase  Correct  Programmed  Reader  Response  Rates 


During 

After 

Rank  of 

Rank  with 

Child 

Phase 

Phase 

d 

D 

Less 
Frequent 
Sign 

Shaupe 

3.08 

1.92 

-1.16 

"  '  16 

-lo 

Smith,  J. 

2.31 

3.46 

+1.15 

7 

Bass 

5.94 

5.74 

-.20 

2 

2 

Fog arty 

5.23 

4.09 

-1.14 

8 

8 

Howard 

4.47 

6.65 

+2.23 

21 

Ford 

3.53 

.74 

-2.84 

26 

26 

Godbolt 

3.36 

4,61 

+1.25 

12 

Hines 

5.23 

6.15 

+  .92 

7 

Nattiel 

4.71 

7.37 

+2.66 

23 

Young 

5.87 

10.10 

+4.32 

30 

Allen 

1.67 

7.32 

+5.65 

32 

Carver 

6.17 

7.47 

+1.30 

14 

Hathway 

6.37 

8.57 

+1.84 

19 

Langston 

7.55 

10.82 

+3.27 

28 

Jones 

7.49 

6.73 

-.76 

6 

6 

Wright 

11.10 

8.95 

-2.15 

20 

20 

Jeffcoat 

7. S3 

2.52 

-5.11 

31 

3.1 

Fog arty 

12.57 

15.76 

+3.19 

27 

Beals 

4.19 

5.97 

+1.78 

17.50 

Lee 

3.97 

5.30 

+1.33 

15 

Ross 

6.24 

7.45 

+1.21 

11 

Somese 

6.86 

8.64 

+1.78 

17.5 

Cray 

4.11 

4.35 

+  .24 

3 

Hardwic 

3.12 

5.92 

+2,80 

25 

Johnson 

3.39 

2.84 

-.55 

5 

5 

Webb 

4.22 

5.84 

+1.26 

13 

Railey 

2,43 

2.76 

+  .33 

4 

Sperring 

9.20 

9.21 

+  .01 

1 

Taylor 

6.31 

4.35 

-1.46 

16 

16 

Bruce 

7.25 

9.99 

+2.74 

2  4 

Hague 

15.20 

11.34 

+3.86 

29 

Howell 

5.38 

7.83 

+2.45 

22 

T  =  124* 
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TABLE  XVI 

A  Wilcoxon  Signed  Rank  Test  for  Differences  Between  During 

Phase  Incorrect  Programmed  Reader  Response  Rates  and 

After  Phase  Incorrect  Programmed  Reader  Response  Rates 


During 

After 

Rank  of 

Rank  with 

Child 

Phase 

Phase 

d 

D 

Less 
Frequent 
Sign 

Shaupe" 

.1.1 

.07 

'  -.02 

"6.5 

Smith,  J, 

.04 

.10 

+  .06 

16.0 

Bass 

.11 

.21 

+  .10 

20.5 

Fog arty 

.07 

.04 

-.03 

10.5 

10.5 

Howard 

.28 

.06 

-.22 

28 

28.0 

Ford 

.42 

.40 

-.02 

6.5 

6.5 

Godbolt 

.33 

.15 

-.18 

26 

26.0 

nines 

.13 

.19 

+  .06 

16 

Nattiel 

.21 

.01 

-.20 

21 

27.0 

Young 

.10 

.02 

-.08 

19 

19.0 

Allen 

.11 

.06 

-.05 

13.5 

13.5 

Carver 

.08 

.24 

+  .16 

22.50 

Hathway 

.21 

.05 

-.16 

22.5 

22.5 

Lang s ton 

.12 

.05 

-.07 

18.0 

18.0 

Jones 

.60 

1.45 

+  .85 

32 

Wright 

.16 

.33 

+  .17 

24.5 

Jef fcoat 

.15 

.17 

+  .02 

6.50 

Fogarty 

.15 

.14 

-.01 

2.0 

2.0 

Beals 

.21 

.24 

+  .03 

10.5 

Lee 

.13 

.14 

+  .0.1 

2.0 

Ross 

.16 

.17 

+  .01 

2.0 

Somese 

.08 

.13 

+  .05 

13.5 

Cray 

.18 

.50 

+  .32 

31 

Hardwic 

.14 

.24 

+  .10 

20.5 

Johnson 

.21 

.27 

+  .06 

5.0 

Webb 

.20 

.50 

+  .30 

29.0 

Railey 

.04 

.02 

-.02 

6.5 

6.5 

Sperring 

.14 

.63 

+  .49 

31 

Taylor 

.09 

.11 

+  .02 

6.5 

Bruce 

.01 

.03 

+  .02 

6.5 

Hague 

.18 

.01 

-.17 

24.5 

24.5 

Howell 

.04 

0 

-.04 

12 

12.0 
T  =  216.5* 
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5  6 
response  reliably  decelerates  the  ignored 
incorrect  response  rate 

7)   the  withdrawal  of  a  subsequent  event  following 
the  incorrect  Programmed  Reader  response 
does  not  reliably  affect  the  ignored  correct 
response  rate 

3)   the  termination  of  an  arrangement  made 

following  correct  Programmed  Reader  responses 
is  associated  with  a  subsequent  acceleration 
of  correct  Programmed  Reader  response  rate 

9)   the  removal  of  an  arrangement  made  following 
incorrect  Programmed  Reader  responses  is 
associated  with  a  maintence  of  incorrect 
Programmed  Reader  response  rate. 


Chapter  ITI 

DISCUSSION 

The  issues  with  which  this  dissertation  is 
concerned  may  be  pooled  into  three  major  areas: 

1)  the  demonstrated  utility  of  free  operant 
techniques  in  the  measurement  and  analysis 
of  programed  curricular  material 

2)  the  effects  of  reinforcement  on  programed 
instruction  response  rate 

3)  the  concurrent  nature  of  correct  and  incorrect 
programed  instruction  performance. 

The  Utility  of  Free  Operant  Techniques 

As  was  pointed  out  in  the  Review  of  Related 
Research  concerning  the  evaluation  of  programed  instruction 
performance,  there  is  a  remarkable  lack  of  precision  and 
standardization  in  the  analysis  and  evaluation  of  programed 
instruction  material.   Indeed,  in  most  cases,  performance 
on  the  programed  material  itself  is  rarely  used  as  the 
dependent  variable  in  the  reported  studies. 

The  important  difference  in  using  free  operant 
techniques  in  the  analysis  of  programed  instruction 
performance  is  primarily  in  the  use  of  direct,  continuous 
recording  of  all  pupil  performance. 

57 
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Markle  (1967),  Mager  (1961)  and  Trow  (1963) 
emphasize  the  importance  of  using  measurable  behavioral 
objectives  in  the  evaluation  of  any  performance.   In  the 
usual  situation,  however,  the  response  required  in  the 
curricular  material  and  the  response  required  in  the  evaluation 
are  not  the  sane  response.   It  is  obvious  that  the  researcher 
cannot  evaluate  directly  one  response  by  measuring  another. 
He  must  be  able  to  record  directly  the  responses  being 
observed  if  he  is  to  make  meaningful  statements  about  those 
responses.   In  this  study,  all  responses  in  the  Programmed 
Reader  and  in  the  Diagnostic  Test  were  continuously  and 
directly  recorded.   This  recording  procedure  allows  statements 
to  be  made  regarding  the  compatibility  of  those  two  portions 
of  the  Sullivan  Reading  Program. 

The  direct  continuous  recording  also  allows  a 
precise  statement  concerning  the  relationship  between 
correct  and  incorrect  performance  rates.   Research  reports 
(Holland,  1965)  traditionally  report  only  percent  correct 
and  percent  incorrect,  forcing  the  two  measures  to  be 
complementary  and  pooling  them  over  time.   This  results  in 
a  tremendous  loss  of  sensitivity  to  behavior  change  (Lindsley, 
1967;  Holzsohuh  and  Dobbs,  1966;  Koenig,  1967).   The 
continuous,  direct,  daily  performance  rates  give  both  the 
teacher  and  the  student  immediate  knowledge  of  performance 
in  all  phases  of  the  programed  instruction  performance. 

In  this  study,  the  direct  continuous  recording 
of  programed  instruction  performance  rate  demonstrated 
that: 
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1)  performance  rates  on  the  Diagnostic  Test 
were  reliably  different  from  performance 
rates  on  the  Programmed  Reader 

2)  correct  Programmed  Reader  response  rates 
ware  reliably  different  from  incorrect 
Programmed  Reader  response  rates. 

In  addition,  a  traditional  pre test-posttest 
evaluation  of  the  Placement  Test  indicated  that  Placement 
Test  performance  was  not  a  reliable  index  of  where  the 
child  performs  in  the  Programmed  Reader. 

The  Effects  of  Reinforcement  on  Programmed  Reader  Performance 

The  question  of  reinforcement  in  programed 
instruction  has  traditionally  bean  discussed  from  a 
theoretical  point  of  view.   Bypassing  the  theory,  which  is 
not  relevant  for  this  discussion,  and  examining  the  experimental 
methodology  found  in  relevant  reports  in  the  literature, 
it  becomes  again  apparent  that  only  rarely  is  the  dependent 
variable  in  such  studies  pupil  performance  (Johnson,  1969). 
Generally,  a  criterion  is  administered,  either  one  made 
especially  for  the  purpose,  or  one  of  the  standardized 
tests  commercially  available.   In  no  case  did  the  writer 
observe  a  direct  continuous  measure  of  pupil  performance 
rate  in  the  within-subject  design  type  of  experiment.   The 
between-subject  design  further  reduces  the  likelihood  of 
those  reported  studies  finding  change,  should  there  be  one, 
thus  increasing  the  probability  of  Beta  error.   The  within- 
subject  design  has  no  such  sensitivity  drawbacks.   Studies 
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utilizing  thewithin-subject  design  with  free  operant  recording 

techniques  have  been  able  to  show  not  only  that  a  change 

occurred  with  respect  to  some  experimental  variable  (i.e., 

the  presentation  of  a  subsequent  event  following  a  response) , 

but  they  were  able  to  show  the  precise  daily  fluctuations 

of  response  rate  associated  with  the  experimental  conditions 

in  effect  (Johnson,  1967a,  1967b;  Koenig,  1967;  Haughton, 

1967)  . 

Using  the  above  type  of  design,  this  study 


observed: 


1)  the  presentation  of  an  experimentally 
selected  subsequent  event  following  the 
correct  Programmed  Reader  response  alone 
accelerated  correct  Programmed  Reader 
response  rate 

2)  the  withdrawal  of  a  subsequent  event  following 
each  incorrect  Programmed  Reader  response 
alone  decelerated  incorrect  Programmed  Feader 
response  rate 

3)  simultaneous  presentation  and  withdrawal 
conditions  both  accelerated  correct  Programmed 
Reader  response  rate  and  deceler-ated  incorrect 
Programmed  Reader  response  rate 

4)  the  presentation  of  a  subsequent  event  following 
the  correct  Programmed  Reader  response 
decelerated  incorrect  Programmed  Reader 
response  rate 

5)  the  withdrawal  of  a  subsequent  event  following 
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the  incorrect  Programmed  Reader  response 
did  not  reliably  affect  correct  Programmed 
Reader  response  rate. 
This  study  also  examined  the  effect  on  subsequent 

pupil  performance  of  terminating  the  reinforcement  condition. 

The  results  are  most  interesting: 

1)  the  removal  of  an  acceleration  consequence 
following  correct  Programmed  Reader  responses 
accelerated  subsequent  correct  Programmed 
Reader  response  rate 

2)  the  removal  of  a  deceleration  consequence 
following  the  incorrect  Programmed  Reader 
response  did  not  reliably  affect  subsequent 
incorrect  Programmed  Reader  response  rate. 

This  result  is  remarkably  similar  to  an  effect 
experimentally known  as  "behavioral  contrast"  (Reynolds, 
1961) .   There  are,  however,  certain  dissimilarities  present. 
Tn  behavioral  contrast,  the  effect  is  noted  following  the 
acquisition  of  a  discrimination.   In  the  present  situation, 
no  experimental  discrimination  was  trained.   However, 
response  differentiation  between  correct  and  incorrect 
Programmed  Reader  responses  was  established.   Whether,  in 
the  present  situation,  this  response  differentiation  provides 
the  conditions  for  the  necessary  stimulus  discrimination 
to  occur  is  not  clear.   Proper  schedule  manipulations  to 
replicate  the  phenomena  within  each  subject  were  not 
performed.   It  can  only  be  noted  that  this  is  a  behavioral- 
contrast-like  phenomena  and  deserves  a  thorough  systematic 
investigation. 
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Such  an  effect  has  been  noted  by  other  researchers. 
Lindsley  (1969)  ,  Koenig  (1969)  ,  and  O'Brien,  Azrin,  and  Benson 
(1969)  report  observing  the  acceleration.   Lindsley  calls  the 
phenomena  "after  effect,"  which,  on  the  basis  of  present 
experimental  evidence,  is  probably  a  more  accurate  descriptive 
term  than  is  "behavioral  contrast." 

For  the  teacher,  this  means  that  a  subsequent 
event  may  be  arranged  for  use  as  a  tool  to  investigate 
a  rate  change,  and  need  not  necessarily  be  used  to  maintain 
that  change.   In  fact,  it  appears  that  any  particular 
arrangement  may  serve  to  limit  operant  strengthening  if 
left  in  effect  too  long.   More  research  will  be  needed  to 
determine  the  guidelines  for  terminating  accelerating 
arrangements  at  the  optimal  time  for  maximal  maintenance 
of  the  movement. 

The  Concurrent  Nature  of  Correct  and  Incorrect  Responses 

Nowhere  in  the  literature  did  this  researcher 
encounter  an  experimental  discussion  of  the  behavioral 
relationship  between  correct  and  incorrect  response  rate 
in  programed  instruction  performance.   Glaser  (1965)  and 
Holland  (1965)  both  provide  extended  disucssions  on  error 
responding  and  error  "rate,"  in  terms  which  are  actually 
not  rates  but  absolute  numbers  or  oercents.   Markel  (1967), 
writing  on  the  empirical  aspects  of  program  evaluation, 
presents  a  similar  analysis. 

This  study  reports  not  only  the  effects  of 
arranged  subsequent  events  for  correct  Programmed  Reader 
responses  and  for  incorrect  Programmed  Reader  responses, 
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but  it  reports  the  effects  of  an  arranged  subsequent  event  for 
correct  Programmed  Reader  responses  on  incorrect  Programmed 
Reader  response  rate  and  the  effects  of  an  arranged  subsequent 
event  for  incorrect  Programmed  Reader  responses  on  correct 
Programmed  Reader  response  rate. 

The  changes  observed  indicate  that  correct 
Programmed  Reader  responses  and  incorrect  Programmed 
Reader  responses  function  as  concurrent  operants  (Catania, 
1966) .   Ferster  and  Skinner  (1957)  define  concurrent  operants 
as  "two  or  more  responses,  of  different  topography  at  least 
with  respect  to  locus,  capable  of  being  executed  with 
little  mutual  interference  at  the  same  time  or  in  rapid 
alternation,  under  the  control  of  separate  programming 
devices."   Educationally,  this  means  that  correct  and 
incorrect  responses  are  not  complementary  movements  with 
respect  to  their  contingencies.   This  further  means  that 
measures  forcing  complementarity  (e.g.,  percent  and 
absolute  number)  are  not  only  insensitive  to  behavior 
change,  but  inappropriate  for  precise  and  accurate  statements 
concerning  correct  and  incorrect  responses. 

The  great  majority  of  research  available  on  the 
nature  of  concurrent  operants  has  been  done  with  non-human 
vertebrates,  chiefly  the  monkey,  pigeon,  and  rat.   One 
central  experimental  problem  in  studying  the  exact  nature 
of  concurrent  schedules  of  reinforcement  is  the  establishment 
of  the  independence  of  the  concurrent  operants  in  question, 
whether  they  are  corapatiable  or  incompatible.   Procedures 
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to  insure  this  independence  are  introduced  primarily  to 
avoid  concurrent  superstition  effects  (Catania,  1965, 
Ferster  and  Skinner,  1957).   Correct  and  incorrect  concurrent 
responses,  even  though  they  are  technically  incompatable, 
viz,  they  can't  occur  at  the  same  place  at  the  same  time, 
are  not  independent,  as  shown  by  the  data  presented  in 
Tables  XI  and  XII.   Specif ically,  the  consequation  of  the 
correct  response  results  in  a  simultaneous  deceleration  of 
incorrect  response  rate,  but  the  deceleration  of  an  incorrect 
response  has  no  noticeable  effect  on  correct  response 
rate. 

This  study  employed  daily  plotting  of  correct  and 
incorrect  response  rates.   This  allowed  only  a  coarse 
grain  analysis  of  the  concurrent  operants  and  therefore  no 
local  interactions  (variations  in  performance  rate  as  a 
function  of  switching  from  one  resoonse  to  the  other) 
could  be  observed.   The  only  statement  that  can  be  made 
concerning  correct  and  incorrect  responses  in  this  study 
is  that  they  appear  to  be  concurrent  operants.   Recordings 
of  individual  responses  will  be  necessary  to  uncover  further 
information  concerning  the  exact  behavior  of  the  operants 
with  respect  to  each  other  for  various  schedules  of 
reinforcement. 


Chapter  IV 


SUMMARY 


There  are  apparently  no  published  reports  of  the 
experimental  analysis  or  evaluation  of  curricular  materials 
in  which  pupil  performance  rate,  directly  and  continuously 
recorded,  was  used  as  a  dependent  variable.   The  present 
study  applied  free  operant  methods  to  the  analysis  of  the 
performance  of  59  reading  disabled  children  en  the  Programmed 
Reader  of  the  Sullivan  Reading  Program.   A  within-subject 
design  allowed  an  experimental  analysis  of  each  individual 
child's  performance.   In  addition,  parametric  and  non- 
parametric  statistical  analyses  were  performed  on  the 
grouped  data. 

The  extreme  sensitivity  of  the  direct  and 
continuous  recording  of  pupil  performance  rate  as  a  dependent 
variable  in  the  analysis  of  programed  materials  was  demonstrated. 
Further,  the  efficiency  of  various  contingency  arrangements 
was  examined.   In  all  conditions,  the  presentation  and/or 
withdrawal  of  a  subsequent  event  produced  an  observable 
change  in  performance  rate. 

The  results  of  the  experimental  procedures  used 
in  the  study  were  discussed  in  the  light  of  their  strong 
similarity  to  results  observed  with  non-human  subjects. 
In  particular,  the  concurrent  nature  of  correct  and  incorrect 
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responses  was  discussed,  and  the  need  for  further  research 
to  uncover  local  interactions  for  various  contingency 
arrangements  was  pointed  out.   The  study  also  noted  a 
post-reinforcement  condition  (after  effect)  similar  to 
the  behavioral  contrast  seen  experimentally  with  animals. 


APPENDICES 


APPENDIX  A 

A  Sample  of  the  Programmed  Reader 

from  Series  I  of  the 

S ullivan  Reading  Prog ram 
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the  big  car. 
The  red  car  is    f sler  than  the  big  car. 


Jack  and  Jill  are  dre.;sed  up 

to  visit  Sam  and  Ann.     Jill  has  en 

a  red  wig  and  black  glasses. 


rod 
black 


whiskers. 


s  stars  on  it. 


Jack  and  his  sister  have  a  present 
to  give  to  Sam  and  his  sister. 


This  is  the  present. 
Is  it  a  banana? 


93 
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Jill 


yes 


Jack's 


y<?s 


Jack  end  Jill  are  with  Sam  and  Ann.      V/hei 
Jack  and  Jill  rang  (he  bell,  Ann  let  them  in. 


Which  girl  has  on  glasses? 
Did  Ann  get  the  present? 


Jill 

Ann 
yes 


Ann  thanked  Jack  and  Jill.     Then  Sam 
passed  a  dish  of  mints.     Jack  had  seven 
mints,  Jiil  had  six,  and  Sam  had  ten. 


....  .      /  Sam's 

Jill  is  <f  sister. 

^  Jack's 


Are  Jack  and  Jill  visiting  Sarn  and  Ann? 


yes 
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I  am  thinking  of  a  thing  that  is  glass 
and  has  jam  in  it. 


<car. 
jar. 


I  am  thinking  of  a  thing  that  has 
a  stamp  and  printing  on  it. 


Itisa/ 


ladder, 
letter. 


I  am  thinking  of  the  part  of  my  arm 
that  has  fingers  on  it. 

I  am  thinking  of  my   and. 


23 


The  tilings  I  am  thinking  of  are  part 
of  a  bird.  A  bird  can  flap  them, 
it  has  !o  flap  ihern  to  fly. 

I  am  thinking  of    ings. 
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APPENDIX  B 

A  Sample  Diagnostic  Test  from 
the  Programmed  Reader 


7  3 


TEST   6 


K 


ft 


a| r  ac r  a    st .  r 


an  rrn 


Sam  went  to  the    p rk    and  sat  in  the 

gr ss.     The  grass  left  a    m rk    on  his  pants. 


A  man  passed  by  and  fed     p tl    of  a  cracker 

to  the  birds. 

Sam  let  the  birds  sit  on  his    . rm. 

yes 
Are  Sam's  arms  part  of  him? 


, 


Lj 


APPENDIX  C 

A  Sample  of  the  Placement  Test 
of  the 
Sullivan  Reading  Program 
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Sample  Pago 


Am  I  an  ant? 


yes 


I  am  a<^ 


Am  I  fat? 


hat. 
cat. 

yos 
no 


I  am  a  fat jat. 


<man. 
pan. 
can. 


it  is  a  tin  c n. 
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TEST  1 


Tills  is  Sam.    This  is  Ann. 
Sam  and  Ann  can  sing. 


Is  Ann  singing? 


yes 


Is  Sam  singing? 


yes 


Is  Sam  siding? 


yes 


.hat 

Sam  has  a  <*"  on. 

^mat 


Sam  has  a  fan  in  his<^ 


hat. 
hand. 


That  is  a   p n  in  Ann's  h nd. 
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TEST  1 


hat 
mat 
cat 


cm 

fan 
tan 


pan 

pin 
pig 


^ 


dish 
'— '-  I       ship 


maps 


an     nt 


a  h nd 
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TEST  2 


bat 
bag 
pig 


rat 
mat 

cat 


kitchen 

kitten 

mitten 


y, 


a     — ing 


a  dr 


3 


trip 
rip 
drip 


l-ps 


.icken 


APPENDIX  D 

One  Complete  Precision  Teaching 
Project  for  one  Response 
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